Motion estimation is an effective inter-frame coding technique to exploit temporal redundancy in video sequences. Motion-compensated inter-frame coding has been widely used in various international video coding standards The motion estimation adopted in various coding standards is often a block-based technique, where motion information such as coding mode and motion vector is determined for each macroblock or similar block configuration. In addition, intra-coding is also adaptively applied, where the picture is processed without reference to any other picture. The inter-predicted or intra-predicted residues are usually further processed by transformation, quantization, and entropy coding to generate compressed video bitstream. During the encoding process, coding artifacts are introduced, particularly in the quantization process. In order to alleviate the coding artifacts, additional processing has been applied to reconstructed video to enhance picture quality in newer coding systems. The additional processing is often configured in an in-loop operation so that the encoder and decoder may derive the same reference pictures to achieve improved system performance.
FIG. 1A illustrates an exemplary system block diagram for an video encoder using adaptive Inter/Intra prediction. For Inter-prediction, Motion Estimation (ME)/Motion Compensation (MC) 112 is used to provide prediction data based on video data from other picture or pictures. Switch 114 selects Intra Prediction 110 or Inter-prediction data and the selected prediction data is supplied to Adder 116 to form prediction errors, also called residues. The prediction error is then processed by Transformation (T) 118 followed by Quantization (Q) 120. The transformed and quantized residues are then coded by Entropy Encoder 122 to form a video bitstream corresponding to the compressed video data. The bitstream associated with the transform coefficients is then packed with side information such as motion, mode, and other information associated with the image area. The side information may also be subject to entropy coding to reduce required bandwidth. Accordingly, the data associated with the side information are provided to Entropy Encoder 122 as shown in FIG. 1A. When an Inter-prediction mode is used, a reference picture or pictures have to be reconstructed at the encoder end as well. Consequently, the transformed and quantized residues are processed by Inverse Quantization (IQ) 124 and Inverse Transformation (IT) 126 to recover the residues. The residues are then added back to prediction data 136 at Reconstruction (REC) 128 to reconstruct video data. The reconstructed video data may be stored in Reference Picture Buffer 134 and used for prediction of other frames.
As shown in FIG. 1A, incoming video data undergoes a series of processing in the encoding system. The reconstructed video data from REC 128 may be subject to various impairments due to a series of processing. Accordingly, various in-loop processing is applied to the reconstructed video data before the reconstructed video data are stored in the Reference Picture Buffer 134 in order to improve video quality. In the High Efficiency Video Coding (HEVC) standard being developed, Deblocking Filter (DF) 130 has been developed to enhance picture quality. The in-loop filter information may have to be incorporated in the bitstream so that a decoder can properly recover the required information.
A corresponding decoder for the encoder of FIG. 1A is shown in FIG. 1B. The video bitstream is decoded by Video Decoder 142 to recover the transformed and quantized residues and other system information. At the decoder side, only Motion Compensation (MC) 113 is performed instead of ME/MC. The decoding process is similar to the reconstruction loop at the encoder side. The recovered transformed and quantized residues and other system information are used to reconstruct the video data. The reconstructed video is further processed by DF 130 to produce the final enhanced decoded video.
In the High Efficiency Video Coding (HEVC) standard, three block concepts are introduced, i.e., coding unit (CU), prediction unit (PU), and transform unit (TU). The overall coding structure is characterized by the various sizes of CU, PU and TU. Each picture is divided into largest CUs (LCUs) or Coding Tree Blocks (CTBs). Each LCU is then recursively divided into smaller CUs until leaf CUs or smallest CUs are reached. After the CU hierarchical tree is done, Inter or Intra prediction is applied to prediction units (PUs) according to partition type. Each PU may be partitioned into one or more smaller blocks (i.e., PUs), such as 2N×2N, 2N×N, N×2N and N×N. Asymmetric partition for prediction units is also allowed. Residues are formed for each PU after applying Inter or Intra prediction. Furthermore, residues are partitioned into transform units (TUs) and two-dimensional transform is applied to the residue data to convert the spatial data into transform coefficients for compact data representation.
The adaptive Inter/Intra prediction shown in FIGS. 1A-B has been widely used in various coding standards. As mentioned above, the residues from the adaptive Inter/Intra prediction are further processed by a two-dimensional transform to exploit the remaining redundancy within the residues. Nevertheless, for some residue data, the two-dimensional transform may not help to improve compression efficiency. An Intra transform skipping scheme is disclosed by Lan et al. for a 4×4 transform unit (“Intra transform skipping”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 9th Meeting: Geneva, CH, 27 Apr.-7 May 2012, Document: JCTVC-I0408). The transform Skip mode for a 4×4 Intra TU by Lan et al. uses the same Intra prediction to form Intra prediction residues. When the transform Skip mode is selected for an underlying block (i.e., 4×4 TU), two-dimensional transform is skipped (i.e., bypassed) for the underlying block on the encoder side. Accordingly, the inverse transform is skipped for the underlying block. Furthermore, the same dequantization process is used as if the block was inverse transformed. In order to use the same dequantization process, the inverse transform-skipped block is scaled down by a factor of 32 for the 4×4 TU. A flag for each 4×4 intra TU is sent to indicate if transform is bypassed or not. Two contexts are added to code the flag for Y, U and V TUs. Another flag in the SPS (Sequence Parameter Set) is used to indicate whether transform skipping is enabled or not.
While the transform Skip mode by Lan et al. demonstrates some performance improvement over a conventional coding system that doesn't support transform skipping, it is desirable to further improve the performance by taking into account the characteristics of the Intra prediction residues.